Operational risk back-testing process using quantitative methods

ABSTRACT

Methods, computer-readable media, and apparatuses are disclosed for quantifying risk and control assessments. The risk includes both residual risk and direction of risk. Various aspects of the invention quantitatively compare the risk and control assessments against step-ahead losses using special regression models that are particularly applicable to this kind of data. The empirical comparison may be performed on both loss event frequency and severity in two different and separate dimensions. The empirical comparison may also be performed using losses extracted by even occurrence and event settlement dates in two separate dates.

RELATED APPLICATIONS

This application is a continuation application that claims priority toNon-Provisional application, U.S. Ser. No. 13/547,855, filed Jul. 12,2012 which is incorporated herein by reference in its entirety.

FIELD

Aspects of the embodiments relate to methods, computer readable media,apparatuses, or computer systems that quantify risk and controlassessments and quantitatively compare those risk and controlassessments against losses using regression models.

BACKGROUND

Risk management is a process that allows any associate within or outsideof a technology and operations domain to balance the operational andeconomic costs of protective measures while protecting the operationsenvironment that supports the mission of an organization. Risk is thenet negative impact of the exercise of vulnerability, considering boththe probability and the impact of occurrence.

An organization typically has a mission. Risk management plays animportant role in protecting against an organization's operational risklosses or failures. An effective risk management process is an importantcomponent of any operational program. The principal goal of anorganization's risk management process should be to protect againstoperational losses and failures, and ultimately the organization and itsability to perform the mission.

Within the financial industry, the Basel II Capital Accord requiresfirms to capture key Business Environment and Internal Control Factors(BEICF) that can change its operational risk profile. These factors willmake an organization's risk environments, help align capital assessmentswith risk management objectives, and recognize both improvements anddeterioration in operational risk profiles in a more immediate fashion.To qualify for regulatory capital purposes the use of these factors inan organization's risk measurement framework must meet standards. Overtime, the process and outcomes need to be validated through comparisonto actual internal loss experience and appropriate adjustments made.

Under the United States final rule (issued in November 2007 byinter-agencies including

FRB (Federal Reserve Board) and OCC (Office of the Comptroller of theCurrency) to implement risk-based capital requirements in the UnitedStates for large, internationally active banking organizations), anorganization has flexibility in the approach it uses to conduct itsBEICFs. As such, the methods for conducting comparisons of theseassessments against actual operational loss experience may also vary andprecise modeling calibration may not be practical. It may still beimportant for an organization to perform such comparisons to ensure thatits assessments are current, reasonable, and appropriately factored intothe organization's AMA framework. In addition, the comparisons couldhighlight the need for potential adjustments to the organization'soperational risk management processes.

Back-testing is the comparison of forecasts to realized outcomes. Anyrisk assessment system is considered well calibrated if the (ex-ante)estimated risk assessment measures deviate only marginally from what hasbeen observed ex-post. The challenge is how to quantify the deviationand how to perform the comparison given all the subtle wrinklespresented by operational risks.

Currently, based on industry benchmarking and from regulators, mostorganizations and banks may perform either a qualitative review or asimple trend analysis (comparing trends of risks against trends oflosses to derive subjective opinions and qualitative outputs). Noorganization currently back-tests subjective operational risk andcontrol assessments against objective losses quantitatively and use themodel output to (a) adjust risk-based capital, (b) forecast step-aheadlosses from risk and control assessments and (c) validate the accuracyof the assessments.

BRIEF SUMMARY

Aspects of the embodiments address one or more of the issues mentionedabove by disclosing methods, computer readable media, and apparatusesfor performing operational risk BEICF back-testing process usingquantitative methods for quantifying risk and control assessments andquantitatively comparing against losses using regression models.

According to an aspect of the invention, a computer-assisted method thatprovides quantification of risk and control assessments andquantitatively compares against losses using regression models. Themethod may include the steps of: 1) identifying a set of riskassessments, a set of control assessments, and a set of financiallosses; 2) translating, by a risk management computer system, the set ofrisk assessments and the set of control assessments to a set of riskstates and a set of risk points; and 3) conducting, by a risk managementcomputer system, data analysis and statistical analysis on the set offinancial losses, the set of risk states, and the set of risk points,wherein the data analysis and the statistical analysis includes aqualitative comparison and analysis and a quantitative comparison.Additionally, the method may further comprise the step of back-testingthe set of risk assessments with the set of financial losses. Thetranslation of the set of risk states into the set of risk points may bedone using an exponential scale. Additionally, the set of riskassessments may include both a residual risk and a direction of risk.The data analysis and the statistical analysis may be performed on theset of financial losses for both a financial loss frequency and afinancial loss severity. Also, the data analysis and the statisticalanalysis may be performed on the set of financial losses aggregated bymultiple loss dates. The two different dates may include a lossoccurrence date and a loss settlement date, and can be extended to otherdates such as loss detection date.

According to aspects of the invention, the quantitative comparisonincludes the steps of:

performing statistical analysis such as correlation and regressionanalysis using regression models, estimating a set of regressioncoefficients and a set of correlation coefficients, and assessing theregression model adequacy. The regression model may include linearregression using a transformed scale. The regression model may alsoinclude quantile regression. The regression model may also include count(frequency) regression.

According to another aspect of the invention, the quantitativecomparison includes the steps of: performing correlation analysis usingboth linear and rank correlation, estimating a set of correlationcoefficients, and determining the statistical significance of the set ofcorrelation coefficients. According to another aspect of the invention,risk-based capital may be adjusted using one or more outputs from thequantitative comparison of risk and control assessments with step-aheadoperational losses.

According to another aspect of this invention, an apparatus maycomprise: at least one memory; and at least one processor coupled to theat least one memory and configured to perform, based on instructionsstored in the at least one memory: 1) identifying a set of riskassessments, a set of control assessments, and a set of financiallosses; 2) translating the set of risk assessments and the set ofcontrol assessments to a set of risk states and a set of risk points;and 3) conducting data analysis and statistical analysis on the set offinancial losses, the set of risk states, and the set of risk points.The data analysis and the statistical analysis may include a qualitativecomparison and analysis and a quantitative comparison. The data analysisand the statistical analysis may be performed on the set of financiallosses for both a financial loss frequency and a financial lossseverity. Further, the quantitative comparison may include one or moreof the following: regression analysis using a regression model; andcorrelation analysis using both linear and rank correlation.

According to aspects of the invention, the set of risk assessments mayinclude both a residual risk and a direction of risk. Additionally, theregression model may include one or more of the following: linearregression using a transformed scale, quantile regression, and countregression. Also, the regression analysis may include estimating a setof regression coefficients and a set of correlation coefficients andassessing the regression model adequacy. Furthermore, the correlationanalysis may include estimating a set of correlation coefficients anddetermining the statistical significance of the set of correlationcoefficients.

According to another aspect of the invention, a computer-readablestorage medium storing computer-executable instructions that, whenexecuted, cause a processor to perform a method may comprise: 1)identifying a set of risk assessments that includes residual risk anddirection of risk, a set of control assessments, and a set of financiallosses, wherein the set of financial losses include financial lossfrequency and financial loss severity; 2) back-testing the set of riskassessments with the set of financial losses; 3) translating the set ofrisk assessments and the set of control assessments to a set of riskstates and a set of risk points using an exponential scale; and 4)conducting data analysis and statistical analysis on the set offinancial losses, the set of risk states, and the set of risk points.The data analysis and the statistical analysis may include a qualitativecomparison and analysis and a quantitative comparison. The quantitativecomparison may include one or more of the following: regression analysisusing a regression model; and correlation analysis using both linear andrank correlation.

These and other aspects of the embodiments are discussed in greaterdetail throughout this disclosure, including the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is illustrated by way of example and not limitedin the accompanying figures in which like reference numerals indicatesimilar elements and in which:

FIG. 1 shows an illustrative operating environment in which variousaspects of the invention may be implemented.

FIG. 2 is an illustrative block diagram of workstations and servers thatmay be used to implement the processes and functions of certain aspectsof the present invention.

FIGS. 3A and 3B show a flow chart for quantifying risk and controlassessments and comparing losses in accordance with an aspect of theinvention.

FIGS. 4 through 9B show various illustrative tables for use with exampleembodiments in accordance with aspects of the invention.

DETAILED DESCRIPTION

In accordance with various aspects of the invention, methods,computer-readable media, and apparatuses are disclosed for quantifyingrisk and control assessments. The risk includes both residual risk anddirection of risk. Various aspects of the invention quantitativelycompare the risk and control assessments against step-ahead losses usingspecial regression models that are particularly applicable to this kindof data. The empirical comparison may be performed on both loss eventfrequency and severity in two different and separate dimensions. Theempirical comparison may also be performed using losses extracted byevent occurrence and event settlement dates in two separate dates.

The empirical comparison may help in (1) meeting regulatory requirements(such as compliance to Basel II AMA); (2) validating the risk andcontrol assessments (RCSA) against actual performance; (3) helping inadjusting capital based on dynamic and empirical data; and (4)forecasting losses at a confidence level determined by the statisticalassociation or relationships of RCSA with losses. Minimally,back-testing may be a “validation” function, but it may also be an“analytical” function. The back-testing may also, for example, help withforecasting future operational losses.

Currently, there is a need for an organization or bank to have a formalprocess to compare

Business Environment and Internal Control Factors (BEICF) assessmentsagainst operational losses. Organizations have not established a soundquantitative process to use these results from this analysis to supportthe methodology for making qualitative adjustments to modeled capitalestimates or calibrate the adjustment range. RCSA's qualitativeadjustment methodology provides a BEICF qualitative adjustment tooperational risk capital between a chosen range (such as −10% and +25%).Generally, organizations do not calibrate the adjustment range regularlyenough. There should be a process established to compare BEICF resultsto actual losses. There should also be a process established for usingthe results of the analysis as an input into deriving an appropriatequalitative adjustment range and applying such adjustments to modeledoperational risk capital estimates. An adjustment range should besupported by empirical analysis to allow reductions to capital. Aspectsof the current invention answer the question, “How good are these riskand control self assessments?” and “Are they good enough that we canadjust the capital or risk decisions based on the risk and control selfassessments?”

According to an aspect of the invention, performing operational riskBEICF back-testing process using quantitative methods may include one ormore of the following steps: 1) defining the granularity andunit-of-measure for the risks; 2) performing exploratory data analysisand statistical analysis; and 3) summarizing the results.

FIG. 1 illustrates an example of a suitable computing system environment100 that may be used according to one or more illustrative embodiments.The computing system environment 100 is only one example of a suitablecomputing environment and is not intended to suggest any limitation asto the scope of use or functionality of the invention. The computingsystem environment 100 should not be interpreted as having anydependency or requirement relating to any one or combination ofcomponents shown in the illustrative computing system environment 100.

The invention is operational with numerous other general purpose orspecial purpose computing system environments or configurations.Examples of well known computing systems, environments, and/orconfigurations that may be suitable for use with the invention include,but are not limited to, personal computers, server computers, hand-heldor laptop devices, multiprocessor systems, microprocessor-based systems,set top boxes, programmable consumer electronics, network PCs,minicomputers, mainframe computers, distributed computing environmentsthat include any of the above systems or devices, and the like.

With reference to FIG. 1, the computing system environment 100 mayinclude a computing device 101 wherein the processes discussed hereinmay be implemented. The computing device 101 may have a processor 103for controlling overall operation of the computing device 101 and itsassociated components, including RAM 105, ROM 107, communications module109, and memory 115. Computing device 101 typically includes a varietyof computer readable media. Computer readable media may be any availablemedia that may be accessed by computing device 101 and include bothvolatile and nonvolatile media, removable and non-removable media. Byway of example, and not limitation, computer readable media may comprisea combination of computer storage media and communication media.

Computer storage media include volatile and nonvolatile, removable andnon-removable media implemented in any method or technology for storageof information such as computer readable instructions, data structures,program modules or other data. Computer storage media include, but isnot limited to, random access memory (RAM), read only memory (ROM),electronically erasable programmable read only memory (EEPROM), flashmemory or other memory technology, CD-ROM, digital versatile disks (DVD)or other optical disk storage, magnetic cassettes, magnetic tape,magnetic disk storage or other magnetic storage devices, or any othermedium that can be used to store the desired information and that can beaccessed by computing device 101.

Communication media typically embodies computer readable instructions,data structures, program modules or other data in a modulated datasignal such as a carrier wave or other transport mechanism and includesany information delivery media. Modulated data signal is a signal thathas one or more of its characteristics set or changed in such a manneras to encode information in the signal. By way of example, and notlimitation, communication media includes wired media such as a wirednetwork or direct-wired connection, and wireless media such as acoustic,RF, infrared and other wireless media.

Computing system environment 100 may also include optical scanners (notshown). Exemplary usages include scanning and converting paperdocuments, e.g., correspondence, receipts, to digital files.

Although not shown, RAM 105 may include one or more are applicationsrepresenting the application data stored in RAM memory 105 while thecomputing device is on and corresponding software applications (e.g.,software tasks), are running on the computing device 101.

Communications module 109 may include a microphone, keypad, touchscreen, and/or stylus through which a user of computing device 101 mayprovide input, and may also include one or more of a speaker forproviding audio output and a video display device for providing textual,audiovisual and/or graphical output.

Software may be stored within memory 115 and/or storage to provideinstructions to processor 103 for enabling computing device 101 toperform various functions. For example, memory 115 may store softwareused by the computing device 101, such as an operating system 117,application programs 119, and an associated database 121. Alternatively,some or all of the computer executable instructions for computing device101 may be embodied in hardware or firmware (not shown). Database 121may provide centralized storage of risk information including attributesabout identified risks, characteristics about different risk frameworks,and controls for reducing risk levels that may be received fromdifferent points in system 100, e.g., computers 141 and 151 or fromcommunication devices, e.g., communication device 161.

Computing device 101 may operate in a networked environment supportingconnections to one or more remote computing devices, such as branchterminals 141 and 151. The branch computing devices 141 and 151 may bepersonal computing devices or servers that include many or all of theelements described above relative to the computing device 101. Branchcomputing device 161 may be a mobile device communicating over wirelesscarrier channel 171.

The network connections depicted in FIG. 1 include a local area network(LAN) 125 and a wide area network (WAN) 129, but may also include othernetworks. When used in a LAN networking environment, computing device101 is connected to the LAN 125 through a network interface or adapterin the communications module 109. When used in a WAN networkingenvironment, the server 101 may include a modem in the communicationsmodule 109 or other means for establishing communications over the WAN129, such as the Internet 131. It will be appreciated that the networkconnections shown are illustrative and other means of establishing acommunications link between the computing devices may be used. Theexistence of any of various well-known protocols such as TCP/IP,Ethernet, FTP, HTTP and the like is presumed, and the system can beoperated in a client-server configuration to permit a user to retrieveweb pages from a web-based server. Any of various conventional webbrowsers can be used to display and manipulate data on web pages. Thenetwork connections may also provide connectivity to a CCTV orimage/iris capturing device.

Additionally, one or more application programs 119 used by the computingdevice 101, according to an illustrative embodiment, may includecomputer executable instructions for invoking user functionality relatedto communication including, for example, email, short message service(SMS), and voice input and speech recognition applications.

Embodiments of the invention may include forms of computer-readablemedia.

Computer-readable media include any available media that can be accessedby a computing device 101. Computer-readable media may comprise storagemedia and communication media. Storage media include volatile andnonvolatile, removable and non-removable media implemented in any methodor technology for storage of information such as computer-readableinstructions, object code, data structures, program modules, or otherdata. Communication media include any information delivery media andtypically embody data in a modulated data signal such as a carrier waveor other transport mechanism.

Although not required, various aspects described herein may be embodiedas a method, a data processing system, or as a computer-readable mediumstoring computer-executable instructions. For example, acomputer-readable medium storing instructions to cause a processor toperform steps of a method in accordance with aspects of the invention iscontemplated. For example, aspects of the method steps disclosed hereinmay be executed on a processor on a computing device 101. Such aprocessor may execute computer-executable instructions stored on acomputer-readable medium.

Referring to FIG. 2, an illustrative system 200 for implementing methodsaccording to the present invention is shown. The system 200 may be arisk management system in accordance with aspects of this invention. Asillustrated, system 200 may include one or more workstations 201.Workstations 201 may be local or remote, and are connected by one ofcommunications links 202 to computer network 203 that is linked viacommunications links 205 to server 204. In system 200, server 204 may beany suitable server, processor, computer, or data processing device, orcombination of the same. Server 204 may be used to process theinstructions received from, and the transactions entered into by, one ormore participants.

Computer network 203 may be any suitable computer network including theInternet, an intranet, a wide-area network (WAN), a local-area network(LAN), a wireless network, a digital subscriber line (DSL) network, aframe relay network, an asynchronous transfer mode (ATM) network, avirtual private network (VPN), or any combination of any of the same.Communications links 202 and 205 may be any communications linkssuitable for communicating between workstations 201 and server 204, suchas network links, dial-up links, wireless links, and hard-wired links.Connectivity may also be supported to a CCTV or image/iris capturingdevice.

The steps that follow in the figures may be implemented by one or moreof the components in FIGS. 1 and 2 and/or other components, includingother computing devices.

FIGS. 3A and 3B show a flow chart 300 for testing operational riskbusiness environment and internal control factors (BEICF) usingquantitative methods in accordance with an aspect of the invention.There may be many different outputs associated with aspects andembodiments of this invention, which may include, but are not limitedto: regression models that help in loss forecasting as a by-product ofthe comparison process; linking capital adjustments based on empiricalcalibrations; empirically identified lines of business (LOBs) orbusiness units and/or enterprise control functions (ECFs) or corporatefunctions that are correct, such as assessment and realization (ofstep-ahead loss) that are aligned; and business units and/or corporatefunctions that are not correct, such as over-assessed risk orunder-assessed risk.

As illustrated in FIG. 3A, the method may include one or more of thefollowing steps: 1) defining the granularity and unit-of-measure for therisks 310; 2) performing exploratory data analysis and statisticalanalysis 320; and 3) summarizing the results that may be used in riskmeasurement (including capital adjustment) or in risk management 350.

Additionally, the method may include both a method for risk control andassessments 302 and for losses 304. The risk control and assessments 302may include translating to risk state and points 306. As illustrated inFIG. 4, the translation of risk control and assessments to risk stateand risk points may help in (1) translating categorical and ordinalvariables to discrete values and (2) collapsing two dimensions, bothrisk state and direction of risk, into one dimension. FIG. 4 illustratesa table 400 for the translation of risk control and assessments to riskstate and risk points that may list residual risk, direction of risk,risk state, risk points, and aggregate risk points across fourdimensions. People, process, systems and external events may be the fourdimensions. Other dimensions may be utilized without departing from thisinvention.

Table 400 indicates a given risk state for each of the four listeddimensions that are translated into risk points and then the combinedaggregate risk points across the four dimensions are obtained by asimple arithmetic summing of the risk points for the 4 individualdimensions. A different implementation can weigh the dimensionsdifferently to obtain a different aggregate risk points. However, in theexample shown in table 400, all four dimensions are equi-weighted forsimplicity. The residual risk column 402 may include ratings of low,medium, and high (3 levels). The direction of risk column 404 mayinclude directions such as decreasing, stable, and increasing (3levels). Other granular levels (more or less) may be defined withoutdeparting from this disclosure. The risk state column 406 may benumbered from 1 through 9. The risk points 408 may be values based onthe risk in increasing values. For example, the risk points 408 mayincrease from 0.5 to 83 over the 9 different risk states. Other valuesand rankings for the risk state 406 and the risk points 408 may beutilized without departing from this disclosure. The aggregate riskpoints across 4 dimensions 410 may generally be calculated bymultiplying the risk points 408 value by 4. FIG. 5 illustrates a table500 defined by the translation of risk state into risk points using anexponential scale. FIG. 5 specifically exponentially graphs the riskpoints 502 from FIG. 4 along the y-axis for each of the risk states 504along the x-axis. Even in exponential increase of risk points other(either steepened or flattened) curves may be selected apart from theparticular choice demonstrated here without departing from thisdisclosure. Other (non-exponential and say polynomial) translation ofrisk state into risk points may be utilized without departing from thisdisclosure.

As described above, the method may include losses 304. The losses 304may be categorized both by loss date and by first loss occurrence date.Other dates (e.g., loss detection date) may be used without departingfrom this disclosure. Additionally, in block 310, the method/process mayinclude defining the granularity and unit-of-measure for the risks. Theunit-of-measure may include enterprise (that may include people,process, systems, and external events) and business units vs. corporatefunctions. Hence it is up to the implementation to back-test either theaggregated enterprise losses or any or all of the individual people,process, systems, and external losses. Similarly, enterprise losses arecomprised of losses suffered by individual business units and enterprisecontrol functions and it is up to the implementation to back-test eitherthe aggregated enterprise losses or any or all of the individualbusiness units or corporate functions. Aspects of the invention maydefine granular units of measure perform the comparisons at granularlevels. For example, the granular levels may include comparisons ofrisks against step-ahead losses for “people risks,” “business unit vs.corporate functions,” “domestic vs. regions,” “business unit-1 vs.business unit-2,” and further on.

As illustrated in FIGS. 3A and 3B, the exploratory data analysis andstatistical analysis step 320 may follow the defining the granularityand unit-of-measure step at block 310. There are many different methodsof exploratory data analysis and statistical analysis 320 that may beutilized without departing from this invention. Generally, the dataanalysis and statistical analysis 320 may include both qualitativecomparison and analysis 322 and quantitative analysis 324.

As illustrated in FIG. 3B, the quantitative analysis 322 may includeregression analysis/modeling 330 and correlation analysis 340. Theregression analysis/modeling 330 may be conducted using both frequencyand severity. The regression analysis 330 may include linear regression,fat-tailed regression or quantile regression, and count (frequency)regression. The linear regression may also include a transformed scaleof either response and/or explanatory variables. Other regression modelsmay be utilized without departing from this invention. The fat-tailedregression models may include quantile regression or linear regressionon log-transformed data for severity based comparison. Other models thathandle zero preponderance well may include zero-inflated negativebinomial and hurdle models used for frequency based comparison of riskand control self-assessments with step-ahead losses.

FIGS. 6A-6D represents typical distributional characteristics exhibitedby operational loss event count data (frequencies) and regression modelsthat are specifically applicable in such cases. Specifically, asillustrated in FIGS. 6A and 6B, the loss event arrival rates (frequencyor counts) exhibit left censored (at zero), zero preponderance and isanything but a normal distribution. Hence standard linear regressionmodels (that assume constant variance, normal errors) are inappropriatefor modeling count data. Furthermore, as illustrated in FIG. 6C, thestandard Poisson count regression (1-parameter) models may not beapplied for data exhibiting fat-tailed, skewed characteristics and zeropreponderance. However, as illustrated in FIG. 6D, the negative binomialand particularly zero-inflated or hurdle models may be found especiallyapplicable for the loss event count regression models and analysis.

Following the regression 330 as performed in any of the above methods,the next step may be estimating regression coefficients and correlationcoefficients 332. Following the estimating regression coefficients andcorrelation coefficients at block 332, the next step may includeassessing model adequacy and iterating 334 to find the best possible fitof appropriate model to the data.

Additionally, following the correlation 340, the next step may beperforming linear and rank correlation 342. Following the linear andrank correlation block 342, the next step may include estimatingcorrelation coefficients and determining statistical significance 344.

Additionally, as illustrated in FIGS. 3A and 3B, following the dataanalysis and statistical analysis 320 step, the next step may besummarizing the results 350. The summarizing results 350 may include acapital adjustment calibration process. Also, the summarizing results350 may include a risk management and RCSA feedback process.

For the summarizing results 350 step, and outputting the results, inanother aspect of this invention, the methods, computer-readable media,and apparatuses may include using the output in the RCSA feedbackprocess and operational risk framework for strengthening the analysis.FIG. 8 illustrates a scatter plot 800 of a quarterly RCSA residual riskpoints versus a second following quarters aggregate losses by FO data.The scatter plot 800 may be graphed in a logarithmic scale. The x-axison the scatter plot 800 may be the log transformed past quarter RCSAaggregate residual risk points 802. The x-axis on the scatter plot 800may span from a 2.5 or low assess risk to a 5.5 or high assessed risk.The choice of the axis span is dependent on the risk-state to risk-pointtranslation shown in FIG. 4. The y-axis on the scatter plot 800 may bethe log transformed aggregate former quarter losses by FO date or thenet new quarterly losses 804.

As illustrated in FIG. 8, the scatter plot 800 may be divided diagonally810 to separate the data. Additionally, in another embodiment, withoutdeparting from the invention, the scatter plot 800 may be divided into a2×2 grid or a 3×3 grid 812. In one example, the scatter plot 800 may beindicated with shades of gray, such as low assessed residual risk buthigh losses realized cell in the upper left corner and high assessedresidual risk but low losses realized in the lower right corner. Thegrids may be created or made using vertical and horizontal lines and/ordiagonal lines.

FIG. 8 illustrates four differing sets of data as defined on the scatterplot 800. The lower left 820 of the scatter plot 800 may define lowresidual risk assessed and extremely low or no losses realized. Thelower right 822 of the scatter plot 800 may define high residual riskassessed and extremely low or no losses realized. The upper left 824 ofthe scatter plot 800 may define medium/high residual risk assessed andhigh losses realized. The upper right 826 of the scatter plot 800 maydefine high residual risk assessed and extremely high losses realized.

For the summarizing results 350 step, and outputting the results, inanother aspect of this invention, the methods, computer-readable media,and apparatuses may include using the output in the capital adjustmentand calibration process for regression analysis. This may be furtherdefined by the calibration of qualitative adjustment based not on BCEIFfactors, but based on the performance of the BEICFs. The followingequation may be utilized to define this capital adjustment process withβ being the regression slope coefficient (obtained from the regressionmodeling).

QA _(i)=β(n _(i)−1)/(n−1) (QA _(max) −QA _(min))+QA _(min)

With:

-   -   QA_(i) defined as qualitative adjustment factor for i^(th) unit;    -   β defined as the regression slope coefficient;    -   n_(i) defined as rank-order of the risk point total for the        i^(th) unit outcome between risk neutral state and highest risk        state. Note: risk-neutral state may be defined as Medium-stable        402 (with medium in residual risk and stable in direction of        risk). Other selections can be made to the risk-neutral state        without departing from this disclosure;    -   n defined as the number of possible risk states (based on risk        points) between risk neutral state and highest risk state;    -   QA_(max) defined as the maximum (ceiling) of the qualitative        adjustment, example 25% or 40%; and    -   QA_(min) defined as the minimum (floor) of the qualitative        adjustment, example 0% or −10% or −25%.

In another aspect of this invention, the methods, computer-readablemedia, and apparatuses may include the output being used in lossforecasting. For example, the out may be calculated using Log (AggregateFO loss amount)=Regression_intercept+Regression_slope_coefficient*(logaggregate risk points)+ε(a constant).

In another aspect of this invention, the methods, computer-readablemedia, and apparatuses may include comparing of risk/control assessmentagainst step-ahead losses. Additionally, in another aspect of thisinvention, the methods, computer-readable media, and apparatuses mayinclude comparing of risk/control assessment against loss frequency andloss severity. The loss frequency may be defined by event count.

In another aspect of this invention, the methods, computer-readablemedia, and apparatuses may include comparing of risk/control assessmentagainst losses by two different dates. The first date may be the firstloss occurrence date and the second date may be the loss settlementdate. The first loss occurrence date may help in comparing with new netlosses. FIG. 7 illustrates a table 700 that shows the comparison ofrisk/control assessment against losses by two different dates. Thechoice of dates used to aggregate losses can be either based onoperational loss event occurrence, detection, settlement, dates.Operational losses such as litigation tend to exhibit multiple impacts(for the same event) that span over long periods of time. Hence lossesaggregated by different dates tend to exhibit non-trivial variations andthis fact may be factored in the selection. The choice of the dateshence may impact the analysis and modeling.

In another aspect of this invention, the methods, computer-readablemedia, and apparatus may include regression models used for thecomparison, comparing beyond statistical correlations. Additionally,operational loss may be used as a responder and risk assessment as apredictor or explanatory variable in the regression models.

In another aspect of this invention, the methods, computer-readablemedia, and apparatuses may include the ability to normalize the datasetfor additional risk management insights. For instance, the comparisonmay be performed on loss for a given amount of money or revenue (forlines of business) and for a give money or expense (for corporatefunctions). Additionally, it may provide the ability to keep track overtime the strength of association of risk assessments with realized(step-ahead) losses.

In another aspect of this invention, the methods, computer-readablemedia, and apparatuses may include reporting of results. FIG. 9Aillustrates a table 900 of Level 1 RCSAs and a correlation analysis byfirst occurrence data (loss severity vs. residual risk). FIG. 9Billustrates a table 920 of Level 1 RCSAs and a severity regressionanalysis by first occurrence date with residual risks. Both FIGS. 9A and9B list consecutive RCSA cycles 902 in quarters, such as Q1, Q2, and Q3.Both FIGS. 9A and 9B also list the risk type 904 for each quarter, suchas Aggregate. FIG. 9A may include a linear correlation for example,Pearson's linear correlation 906 that may include untransformed data andlog transformed data for both the coefficient and p-value. Additionally,FIG. 9A may include rank correlations 908 using both Spearman's Rho andKendall's Tau for the coefficient and p-value. FIG. 9B may include logtransformed data 922 for an adjusted R-squared, the coefficient, and thep-value. In FIG. 9B, item 906 may report the magnitude of the linearcorrelation of risk and control self-assessment with a step-ahead lossesand they tend to be closer to 1.0 and away from 0.0 if theself-assessments turn out to be rather accurate. Similarly, item 908 inFIG. 9A reports magnitude of rank correlation. Additionally, item 922 inFIG. 9B reports sample regression modeling results including regressionslope (indicated by Adjusted R-square), regression coefficient andp-value. Similar reporting can be used for frequency (count) dataregression modeling. Other reporting structures and styles may be usedwithout departing from this disclosure.

Additional embodiments of this invention may include a broader andbigger market beyond the domestic United States. Basel II compliance maybe phased with Europe and other North American early pioneers, comparedto other regions/countries. The aspects and embodiments of thisinvention may be utilized within the United States and outside of theUnited States. Even though regional central banks and organizations mayextend the Basel II framework for regulatory compliance and guidelines,by and large, many other countries follow the guidelines set for in theUnited States. Many firms and organizations (even non-banking andnon-financial sector) require operational risk BEICF back-testing. Theconcept of operational risk BEICF back-testing may be industry agnostic,so many other industries and organizations may utilize the operationalrisk BEICF back-testing process as described without departing from thisinvention.

Aspects of the embodiments have been described in terms of illustrativeembodiments thereof. Numerous other embodiments, modifications andvariations within the scope and spirit of the appended claims will occurto persons of ordinary skill in the art from a review of thisdisclosure. For example, one of ordinary skill in the art willappreciate that the steps illustrated in the illustrative figures may beperformed in other than the recited order, and that one or more stepsillustrated may be optional in accordance with aspects of theembodiments. They may determine that the requirements should be appliedto third party service providers (e.g., those that maintain records onbehalf of the company).

We claim:
 1. A computer-assisted method comprising: translating a set ofrisk assessments and a set of control assessments to a set of riskstates and a set of risk points across a plurality of dimensions,wherein the dimensions comprise at least people, process, systems andexternal events; determining a set of aggregate risks comprising acombination of the set of risk points corresponding to each dimension ofthe plurality of dimensions; and conducting, by one or more computingdevices, data analysis and statistical analysis on a set of realizedfinancial losses, the set of risk states, the set of risk points, andthe set of aggregate risks wherein at least one of a risk-based capitaladjustment and a model assessment is determined based on a result of thedata analysis and statistical analysis.
 2. The method of claim 1,further comprising: conducting, by the one or more computing devices, aqualitative comparison and analysis and a quantitative comparison duringthe data analysis and the statistical analysis, wherein the set offinancial losses are defined as step-ahead operational losses.
 3. Themethod of claim 2, wherein the quantitative comparison includes thesteps of: performing, by the one or more computing devices, correlationanalysis using linear correlation and rank correlation, estimating, bythe one or more computing devices, a set of correlation coefficients,and determining, by the one or more computing devices, a statisticalsignificance of the set of correlation coefficients.
 4. The method ofclaim 2, comprising using a set of results as defined from a qualitativecomparison and analysis and a quantitative comparison to adjust capitalof an organization.
 5. The method of claim 1, further comprising:back-testing, by the one or more computing devices, the set of riskassessments with the set of realized financial losses.
 6. The method ofclaim 1, wherein the translation of the set of risk states and the setof risk points is done using an exponential scale.
 7. The method ofclaim 1, wherein the set of risk assessments include a residual risk anda direction of risk.
 8. The method of claim 1, wherein the data analysisand the statistical analysis is performed on the set of financial lossesfor a financial loss frequency and a financial loss severity.
 9. Themethod of claim 1, wherein the data analysis and the statisticalanalysis is performed on the set of financial losses aggregated bymultiple loss dates.
 10. The method of claim 9, wherein the multipleloss dates include a loss occurrence date and a loss settlement date.11. The method of claim 1, further comprising: conducting, by the one ormore computing devices, a qualitative comparison and analysis and aquantitative comparison during the data analysis and the statisticalanalysis, and wherein the quantitative comparison includes the steps of:performing, by the one or more computing devices, statistical analysisincluding correlation and regression analysis using regression models,estimating, by the one or more computing devices, a set of regressioncoefficients and a set of correlation coefficients, and assessing, bythe one or more computing devices, a regression model adequacy.
 12. Themethod of claim 11, wherein the regression models includes linearregression using a transformed scale.
 13. The method of claim 11,wherein the regression models includes quantile regression.
 14. Themethod of claim 11, wherein the regression models includes countregression.
 15. An apparatus comprising: one or more processors; and anon-transitory memory device communicatively coupled to the one or moreprocessors, the non-transitory memory device storing instructions that,when executed by the at least one processor, cause the apparatus to:translate a set of risk assessments and a set of control assessments toa set of risk states and a set of risk points across a plurality ofdimensions; determine a set of aggregate risks, wherein each aggregaterisk of the set of aggregate risks comprises a combination of riskpoints corresponding to a risk state of each dimension of the pluralityof dimensions; and conduct data analysis and statistical analysis on aset of realized financial losses, the set of risk states, the set ofrisk points, and the set of aggregate risks wherein at least one of arisk-based capital adjustment and a regression model assessment isdetermined based on a result of the data analysis and statisticalanalysis.
 16. The apparatus of claim 15, wherein the set of riskassessments include a residual risk and a direction of risk.
 17. Theapparatus of claim 15, wherein a regression model comprises one or moreof the following: linear regression using a transformed scale, quantileregression, and count regression.
 18. The apparatus of claim 15, whereinthe instructions, when executed by the at least one processor, cause theapparatus to: estimate a set of regression coefficients and a set ofcorrelation coefficients; and assess a regression model adequacy.
 19. Anon-transitory computer-readable storage medium storingcomputer-executable instructions that, when executed by a processor,cause the processor to: identify a set of risk assessments that includesresidual risk and direction of risk, a set of control assessments, and aset of realized financial losses, wherein the set of realized financiallosses include financial loss frequency and financial loss severity;translate the set of risk assessments and the set of control assessmentsto a set of risk states and a set of risk points across a plurality ofdimensions, wherein the dimensions comprise at least people, process,systems and external events; determine a set of aggregate riskscomprising a combination of the set of risk points corresponding to eachdimension of the plurality of dimensions; and conduct, by one or morecomputing devices, data analysis and statistical analysis on a set ofrealized financial losses, the set of risk states, the set of riskpoints, and the set of aggregate risks wherein at least one of arisk-based capital adjustment and a model assessment is determined basedon a result of the data analysis and statistical analysis.
 20. Thenon-transitory computer-readable storage medium of claim 19, storingcomputer-executable instructions that, when executed, further cause theprocessor to: determine at least one of a financial loss frequency and afinancial loss severity based, at least in part, on a result obtainedfrom the data analysis and the statistical analysis, wherein the set ofrealized financial losses are from two different dates that include aloss event occurrence date and a loss event settlement date.